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Abstract 

In this paper, a class of general nonlinear programming problems with inequality and equality constraints is discussed. 
Firstly, the original problem is transformed into an associated simpler equivalent problem with only inequality con- 
straints. Then, inspired by the ideals of sequential quadratic programming (SQP) method and the method of system 
of linear equations (SLE), a new type of SQP algorithm for solving the original problem is proposed. At each it- 
eration, the search direction is generated by the combination of two directions, which are obtained by solving an 
always feasible quadratic programming (QP) subproblem and a SLE, respectively. Moreover, in order to overcome 
the Maratos effect, the higher-order correction direction is obtained by solving another SLE. The two SLEs have the 
same coefficient matrices, and we only need to solve the one of them after a finite number of iterations. By a new line 
search technique, the proposed algorithm possesses global and superlinear convergence under some suitable assump- 
tions without the strict complementarity. Finally, some comparative numerical results are reported to show that the 
proposed algorithm is effective and promising. 
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1. Introduction 

In this paper we consider the following nonlinear programming problem 

min fo(x) 

s.t. Mx)<0, ieli := {1,2, . . . (1) 
fi(x) = 0, i € li := {m' + 1, m' + 2, . . . , m}, 
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where /j : R" — > R (i e {0} U I\ U I2) are smooth functions. The feasible set and gradients of problem ([]]) are denoted 
as follows: 

fi:=(ie R" : Mx) < 0, z € h; fi(x) = 0, i e 7 2 }, and g,(jc) := V/Xx), z e {0} U^U 7 2 - 
Sequential quadratic programming (SOP) algorithms have been widely studied by many authors during the past 



5l a 7|], and have been proved highly effective for solving problem (Q3. 



several decades, e.g., Refs. 

SQP algorithms generate iteratively the main search directions by solving the standard quadratic programming (QP) 
subproblem 

min go(x) T d + ^d T Hd 
s.t. fj(x) + gi(x) T d < 0, i G I\, 
fi(x) + gi(x) T d = 0, !€ I 2 , 

where H e R nx " is a symmetric positive definite matrix. Then one performs a line search which is a one dimensional 
minimization problem to determine a steplength, and obtain the next iteration point. 

SQP algorithms may fail since the equality constraints of QP subproblem are hard to be satisfied in the process of 
iteration. Mayne and Polak 1180 propose a new way for overcoming this difficulty. In their scheme, they consider the 
following related family of simpler problem 

min F c (x) := f Q (x) - c 2 /«(*) 

m (2) 

s.t. fi(x) < 0, i e I, 

where I := I\ U I2 and parameter c > 0. Especially, F c (x) = fo(x) if h = 0- We denote the feasible set of problem (f2]i 
by 

Q + := {x e fl" : fox) <0, ie I}. 

Moreover, they prove that the original problem (Q3 is equivalent to problem © when c is sufficiently large but finite. 
Note that problem (f2]i only has inequality constraints, so the corresponding QP subproblem has not equality con- 
straints, and SQP algorithms will be always successful under some suitable conditions. More advantages and further 



applications of this technique can be seen in 



111 



Recently, Guo propose an algorithm for solving problem dl} with I2 — in [l^l. In this algorithm, the initial 
iteration point can be chosen arbitrarily. The main search direction is obtained by solving one QP subproblem and one 
(or two) system(s) of linear equations. The algorithm possesses global and superlinear convergence under some suit- 
able assumptions without the strict complementarity. Furthermore, some comparative numerical results are reported 
to show that the algorithm is effective. 



Inspired by the ideas in J8 , 



1211 . we propose a new SQP algorithm for solving problem ([TJ. First, problem ([!} is 



equivalently transformed into problem (|2} (see Lemma |2~2V In order to overcome the inconstant of QP subproblem, 



2 



we consider a modified QP subproblem 

min VF c (x) T d + \d T Hd 

s.t. fi(x) + gi(x) T d < <p(x), i £ 7 + (x), (3) 

fi(x) + gi(x) T d < 0, ie l~{x), 

where tp{x) := max{0,/Kx), / € /}, 7 + (jc) := {/ e I : fi(x) > 0}, I~(x) := {i e 7 : < 0}. QP subproblem © has 
the following advantages: 

• subproblem (0 always has a feasible solution d = 0. 

• subproblem (0 is a strictly convex program if H is positive definite, so it always has a unique solution. 

• d is a solution of subproblem (O if and only if it is a KKT point of subproblem (|3). 

In order to get the global convergence of the algorithm, the search direction is generated by the combination of 
two directions, which are obtained by solving QP subproblem (0 and a system of linear equations, respectively. For 
overcoming the Maratos effect lljL the higher-order correction direction is generated by solving another system of 
linear equations. The two systems of linear equations have the same coefficient matrices. The superlinear convergence 
is derived under the strong second-order sufficient conditions (SSOSC) without the strict complementarityJVloreover, 
for further comparing the performance of the method of strongly sub-feasible directions (MSSFD) HQ 1121 1 with the 
method of quasi-strongly sub-feasible directions (MQSSFD) 11411 . the technical of MQSSFD is adopted in our new 
algorithm. Finally, some comparative numerical results are reported to show that our new algorithm is promising. The 
main features of the proposed algorithm are summarized as follows: 

• the initial iteration point is arbitrary, and the number of constraints satisfying constraint condition is monotone 
nondecreasing. 

• the objective function of problem (0 is used directly as the merit function. 

• the parameter c is adjusted automatically only for a finite number of times (see Lemma |3~TT i. 

• at each iteration, the search direction is generated by a combination of two directions, which are obtained by 
solving an always feasible QP subproblem and a system of linear equations, respectively. 

• after finite iterations, the iteration points always lie in Q + . 

• under SSOSC without the strict complementarity, the proposed algorithm possesses global and superlinear 
convergence. 

The paper is organized into six sections. In Section [2] our new algorithm and its properties are presented. In 
Sections [3] and |H we show that the proposed algorithm possesses global and superlinear convergence, respectively. 
In Section some comparative numerical results are reported to show that the proposed algorithm is effective and 
promising. Some conclusions about the proposed algorithm are given in Section|6] 

Throughout the paper we use the following notations for a point x e R" and an index subset J Ql 

J fj(x) := {fi(x), i e J}, gJ (x) := { gi (x), i e J], f t (x) := Mx), i e Fix), f&x) := /;(*) - tp(x), i e I + (x), 
I h(x) := {j e h : Mx) = 0}, I(x) := h(x) U I 2 , Z (jc) := (ie/: Mx) = 0). 



2. Description of algorithm 

In this section, we start by giving some basic assumptions for problem (0. 

Assumption 2.1. (i) The functions f(x) (i e {0} U /) are all continuously differentiable. 
(ii) The gradient vectors {gi(x) : i e I(x)} are linearly independent for each x 6 R". 

To update the parameter c in problem 0, the matrices N(x), D(x) and multiplier vector n(x) are defined as follows: 

N(x) = ( gi (x), i e I), n(x) = -(N{x) T N(x) + D{x)y l N(x) 1 g (x), 

' \MxW, iel u (4) 
0, i e / 2 , 



D(x) = diag(D,(x), i € /), D t (x) = 



where p is a positive parameter. 

Note that D,(x) > for all i e I\I(x) in ©. By Assumption ^, lf ii). the following lemma holds immediately. 

Lemma 2.1. Suppose that Assumption \2. 1 1 holds. Then (N( x) T N(x) + D(x)) is nonsingular and positive definite for 
all x 6 R". 

By Lemma [XT] and ©, the relationship between problems ([1} and (f2]i is shown in the following Lemma [2721 Its 



proof can be referred to the one of Lemma 2. l(v) in 111 Oil . 



Lemma 2.2. If c > \m(x)\for all i e then (x,p.) is the KKT point of problem |7]) if and only if (x, X) is the KKT 
point of problem (0, where p and A satisfy 

Pi = Aj, ielu pi = Ai -c, i e I 2 . (5) 

For the iteration point x k and the parameter q of problem ©, QP subproblem Q can be simplified as follows by 
the above notations 

min VF c Ax k ) T d+y T H k d 

2 (6) 
s.t. f{x k ) + gi(x k ) T d < 0, iel. 

When Hk is positive definite, rfj is a solution of subproblem (O if and only if there exists a corresponding KKT 
multiplier vector A k such that 



VF Ct (x k ) + H k d k + £ A k gi(x k ) = 0, 

iel 

f(x k ) + gi{x k ) T d k () <0,A k > 0, A k (fi(x k + gi (x k ) T d k )) = 0, V i € I. 
Since d — is a feasible solution of subproblem (0 and 74 is positive definite, it follows that 

VF Ct (x k ) T d k + ^(d k ) T H k d k < => VF^fdl < 0, 

i.e., rfg is a descent direction of F Ck (x k ) at the iteration point x*. 

Due to (0 and Lemma l2~2l the following lemma holds immediately. 
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k -Clld*H + <p{x k y)Tn t 



Lemma 2.3. If (d^, f(x k )) = (0,0), then x k is the KKT point of problem J2]). Furthermore, ifc k > \p. k \for all i e h, 
then x k is the KKT point of problem |7]). 

Again from (J7), it follows that may not be a feasible direction of problem (f2) at the feasible iteration point 
x k eff. So a suitable strategy must be carried out to generate a feasible direction. Here, taking into account that x k 
may be infeasible, we introduce the system of linear equations to get a unique solution (d\, h k ) 

(8) 

h ) [N j k -Q k ){h ) \ -ciid*n+ itfrya ) 

where 6 R n , w = (1, 1, . . ., If e R m , cr e (0, 1) and 

N k = = (g,( A i e /), Q k = diag(g* = |/(J(*)|(W(J(*) + gi (^) T d k \ + \\d k \\), i € /). 

Then we consider the convex combination of c& and d\ 

d k = (i-A)4+M- 

where /?,<. is the maximal value of f3 e [0, 1] satisfying 

WF Ct (x k ) T d k < 0VF Cl (x k ) T d k Q + ^(/) e . 
Moreover, (fill further implies that f3 k is the optimal solution of linear programming 
max p 

s.t. /3VF Ct (x k ) T d k + (1 -^)VF Ct (jc*) r 4 < 0VF Ck (x k ) T d k Q + ip(x k ) e , 
0<P< 1, 

where the positive parameter 6 < cr. It is obviously that (fTTT i holds for = 0, since (p(x k ) > 0. The above linear 
programming further implies that /3 k > 0, otherwise, x k is the KKT point of problem |Q]) (see Lemma [23b . 

The next lemma shows the solvability of ([8]). Its proof is elementary in view of {z € / : Q k = 0} c I Q (x k ) c /(**). 

Lemma 2.4. Suppose that Assumption YlA\ holds and H k is positive definite. Then T k defined in dSj « nonsingular and 
a unique solution. 



(9) 



(10) 



(11) 



Lemma 2.5. Suppose that Assumption s. l\ holds. Then 

(i) WF Ck (x k ) T d k < -±8(d k ) T H k d k + <p(x k ) e . 

(ii) gi (x k ) T d k < -MW4W + tftfT), V i e I (x k ). 

proof: (i) Since d — is a feasible solution of subproblem (|6), and from (fTTT i. it holds that 

VF Ck (x k ) T d k < OVFc^fd* + <p(x k ) e < -±6(d k ) T H k d k + <p(x k ) e . 



(ii) First, from Q and ©, it follows that 

gi(x k fd k Q < 0, g i (x k ) T d\ = -||4|| - V i e I (x k ). 

Then we obtain that by dTOb 

gi (x k ) T d k < -M\\4\\ + y{x k T) for all i e /„(**). 

□ 

By Lemma 1231 we know that <r is an improved direction. In order to overcome the Maratos effect and avoid the 
strict complementarity condition as well as reduce the computational cost, a suitable higher-order correction direction 
should be introduced by an appropriate approach. Here, we introduce the following system of linear equations to yield 
the higher-order correction direction d\ 



1 \ 
d 


t 


h 

V ) 


V 



-(\\d k Q \Y + rtx k y)m-F(x k + d^ j 



where r e (2, 3) and 



F(x k + d k ) = (/*(** + d k ) - Mx k ) - gi (x k ) T d k , i e I). 



(12) 



(13) 



Note that the term <p(x k ) is introduced in our paper, the relationship between d\ and d\ will be different from the 
traditional form = 0(||4|| 2 ) HQQ, 

the details can be seen in Lemma |4~T1 
We are now ready to present our algorithm for solving problem ([T} as follows. 

Algorithm 2.1. 

Parameters: p,e,y,jo > 0, c_i > 0, p > 1, < 8 < cr, cr, r\, a, a e (0, 1), t e (2,3). 
Data: x° e R", a symmetric positive definite matrix Ho e R nxn 7 and k := 0. 
Step 1. Update parameter q: Compute q (k — 1, 2, . . .) by 



Ck = 



maxjsi, c k -i + y}, if s k > c k - 



Ck-U 



if Sk < Ck-u 



Sk = max{|7r,(x )|, i e I 2 ] + yo- 



(14) 



Step 2. Solve QP subproblem: Solve QP subproblem © to get a solution. If (d k Q , <p(x k )) = (0, 0), then x k is the 
KKT point of problem (T]) and stop; otherwise, go to Step 3. 

Step 3. Solve system of linear equations: Solve (fT2l to get a solution {d\, h^), and let d k = d k . 
Step 4. LeU= 1. (a) If 



F Ck (x k + Uift < F Ck (x k ) + atVF Ct (x k ) T d k +p(l - a)t<p(x k ) , 
fi(x k + td k ) < max{0, <p(x k ) - at(\\d k \\ T + tpixY)}, i e /, 
\r(x k + td k )\ > \r(x k )\, 



(15) 



is satisfied, then let t k = t, and go to Step 7; otherwise, go to (b). 
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(b) Let t := \t. 
If t < e, then go to Step 5; otherwise, repeat (a). 

Step 5. Solve system of linear equations: Solve (0 to get a solution (d k , h k ), and compute d k by ( TTOb and (TTTT) . 
Step 6. Compute steplength ft be the first member of the sequence {1, r\, rf, . . .} such that 

F Ck (x k + td k ) < F Ck {x k ) + atVF Ck (x k ) T d k + p(l - a)t(p(x k ) e , 
• fi(x k + td k ) < max{0, (p(x k ) - at/3 k (\\d k \\ + <p(x k Y)), i e I, (16) 
\r(x k + td k )\ > \r(x k )\, 

and let d k = d k . 

Step 7. Compute a new symmetric positive definite matrix H^+i by some techniques, set x k+l :- + t^d k , k :- 
k + 1, and go to Step 1. 

Lemma 2.6. Suppose that Assumption 12.71 holds. If Alsorithm \2.1\ does not stop at Step 2, i.e., (d k v if(x k )) + (0,0), 
then the line search ( 17(51 1 can be terminated after a finite number of iterations. 

In fact, the above Lemma 12761 shows that Algorithm 12. H is well defined. Moreover, d k is an improved direction in a 
sense by Lemma |231 Therefore, if Algorithm |3T| does not stop at Step 2, i.e., (d k ,tp(x k )) + (0,0), we can always 
get the next iteration point x k+1 from the current iteration point x k according to Lemma |2~61 Furthermore, from the 
mechanism of Algorithm l2.ll the following lemma holds obviously. 

Lemma 2.7. Suppose that Assumption \2~J\ holds. 

(i) If there exists an index ko such that x k " e Q + , then x k € Q + and F Ck (x k+l ) < F Ck {x k )for all k > ko. 

(ii) Ifx k t C2 + andx k+l i Q. + , then ip(x k+l ) < ip(x k ). 

(Hi) The subsets 7~(x*) and I + (x k ) can be fixed, i.e., I~(x k ) = I and I + (x k ) = I + for k large enough. 
3. Global convergence 

In this section, we establish the global convergence of Algorithm 12. II If Algorithm 12. 1 1 stops at x*, it follows 
that x k is the KKT point of problem (T]). Now, we assume that Algorithm l2. ll produces an infinite sequence {x^} of 
iteration points, and prove that each accumulation point x* of {xr } is the KKT point of problem |Q]) under some suitable 
assumptions. For this purpose, the following assumption is necessary. 

Assumption 3.1. (i) The sequence {x k } is bounded. 
( ii) There exist positive constants a and b such that 

a\\d\\ 2 < d T H k d < b\\d\\ 2 , V d e R", V k. (17) 

Denote the active set for QP subproblem © by 

L(x k ) s{iel: f(x k ) + gi (x k ) T d k = 0}. 
7 



Suppose that x* is a given accumulation point of {x k }. In view of I + (x k ), I (x k ) and L(x k ) are subsets of the finite set 
/, by Lemma lZTl iii), we can assume that there exists an infinite index set K such that 

x k -» x\ r(x k ) = r, I + (x k ) = I + , L(x k ) = L, (fix*) -» <p(x*), V keK. (18) 



Lemma 3.1. Suppose that Assumytions \2.1\ and \3 AM ) hold. Then there exists an index k\ > such that q = c*, = c 
for all k > k\. 

The detailed proof of this lemma can be found in [10]. Due to Lemma |3~T1 we assume that q = c for all k in the 
rest of this paper. The results given in the following lemma are very important in the subsequent analysis. 



Lemma 3.2. Suppose that Assumptions \2.1\ and \3.1\ hold. Then 
(i) The sequence [di.}^ is bounded. 

(iij There exists a constant ro > such that \\T7 \\ < rofor all k. 

(Hi) The sequence {d k }™ ={ , f^ljli' $ k }T_i and {h*}^ are all bounded. 

proof: (i) Due to the fact that d = is a feasible solution of subproblem (|6]l and d k Q is an optimal solution, we have 

WF c (x k ) T d k + l -{d k Q ) T H.dl < 0. (19) 

By Assumption 13.11 and the continuity of VF c (x k ), there exists a constant c > such that ||VF c (x*)|| < c for all k. 
Combining ( TT9b with ( TTTb , we get 

-c||4|| + l -a\\d k \\ 2 < 0, 

which implies that {d^} is bounded for all k. 

(ii) Suppose by contradiction that there exists an infinite index set K such that 



HIT 1 1| -> oo, keK. (20) 



Without loss of generality, we assume that there exists an infinite index set K' c K such that 

d\ d*, <p(x k ) ip(x*), h(x k ) I (x*l VkeK', 



x k — » x*, Fir 



Ni -or ) 



where 

N. = ( 8i (x*), i € /), Q* = diag(g; = [hixlMix*) + gi (x*) T d* Q \ + \\dl\\\ i E /). 

It holds that Q* > for all i e /, and Q* > for all i e I\Iq(x*). Similar to the proof of Lemma l2~4l we can conclude 
that F» is nonsingular. So ||r^ '|| — > \\T^ || for k e K', which contradicts ( T20l i. thus the conclusion (ii) holds. 

(iii) Taking into account ©, (O and ((TU), we can obtain the boundedness of {d\}™ =v {d k }™ =l , {d k }™ =l and {h k 2 }™ =l 
by employing the result of parts (i) and (ii). □ 

Similar to the analysis of Lemma 3.3 in J12I1 . we can obtain the following results. 



Lemma 3.3. Suppose that Assumptions \2.1\ and \3 . H are satisfied. Then 

(ij ]im(di., (p{x k )) = (0, 0), lim d\ = lim d k = lim d k = and lim h\ = 0. 

(ii) lim ||jc* +1 - x k \\ = 0. 
k— >oo 

Theorem 3.1. Suppose that Assumptions \2.1\ and \3.1\ hold. Then Algorithm \2.1\ either stops at the KKT point 
of problem (0 after a finite number of iterations or generates an infinite sequence {x k } of points such that each 
accumulation point x* of {x k } is the KKT point of problem (0. Furthermore, there exists an index set K such that 
{(x k , A k ) : k e K] and {(x k ,p k ) : k e K) converge to the KKT pair (x* , A*) of problem (0 and the KKT pair (x*,p*) of 
problem (Q, respectively, where A k = (A k L , Qi\l) and p k — (p k L , Oi\l). 

proof: By Lemma lZTl 'iii). we assume without loss of generality that there exists an infinite subset K such that ( TT~8T > 
holds. Let matrix A* = (gi(x k ), i € L). From Lemma l331l ). it follows that L c Iq(x*) -{iel: f(x*) = 0}, 
which together with Assumption 12. 1 1 shows that AjA,t is nonsingular for keK large enough, since At A», where 
A, = (gj(x*), i e L). 

From (O and Lemma [33? i). we have for keK large enough, 

4 = -(A T k A k y l A T k {VF c (x k ) + H k d k ) -(AlA,y x AlVF c {x*) 4 T L . 

Denote the multiplier vector A* = (/1*,0/\l), then lim A k = A*. Passing to the limit k e K (k — > oo) in (0, it follows 

keK 

that 

VF c (x*) + N*A* = 0, f(x*) < 0, A* > 0, f(x*)A* =0,ie I, 

which shows that {x*,A*) is the KKT pair of problem (0. By the definition of q and Lemma I3T1 we have c > 
max{7r,(jc*) : i e h}. So from Lemma |Z21 we can conclude that (x*,fi*) is the KKT pair of problem ([1]) with 
p* = A% i e L\h; p* - A* - c, i e I 2 ; p* = 0, i e I\L. Obviously, lim(x k ,A k ) = (x*,A*) and lim^*,^*) = (x*,p*). 

keK keK 

The proof is completed. □ 
4. Rate of convergence 

In this section we further discuss the strong and superlinear convergence of Algorithm ^. II For these purposes, we 
make the following assumption. 

Assumption 4.1. ( i) The functions f(x) (i e {0} U /) are all second-order continuously differentiable. 
(ii) The KKT pair (x* , p*) of problem (0 satisfies the strong second-order sufficient conditions, i.e., 

d T V 2 xx L(x*,p*)d > 0, V d e R", d + 0, gi(x*) T d = 0,ie 7, + , 

where V 2 KX L(x*,p*) = V 2 /o(**) + 2 ^ 2 fi(x*), It = [i e h : % > 0} U 7 2 . 

iel 
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Remark 4.1. Similar to the proof of Lemma \272\ we can conclude that (x* , A*) satisfying 

A* = fi*, i eh; A* = ft* +c, i e I 2 (21) 

is the KKT point of problem Moreover, {iel: A* > 0} = {i e I\ : p* > 0} U I2, which implies that KKT pair 
(x*, A") of problem ([3j also satisfies the strong second-order sufficiency conditions, i.e., 

d T l 2 xx L c (x\ A*)d >0,\/de R", dtO, g&x*fd = 0, i e 7+, 

where l 2 xx L c (x\ A") = V 2 F c (x*) + £ A*V 2 f(x*), It = {iel: A] > 0). 

iel 

Under the stated assumptions, we have the following theorem. 



Theorem 4.1. Suppose that Assumptions \2.1\\3.1\ and \4.1\ hold. Then 
(7) lim x k = x* , i.e., Alsorithm \2.1\ is strongly convergent. 

k^>co 

(ii) lim A k = A", lim fi k - p*. 

k— >oo k^>co 



31. 



proof: (i) The proof of this part is similar to the one of Theorem 4.1 in ||3|], and the details can be seen in [ 

(ii) From the proof of Theorem 13 . 1 l and part (i), one can conclude that each accumulation point of sequencees {A k } 
and {//} is the KKT multiplier for problem 0} and problem ([TJ associated with x* , respectively. Togethering with the 
uniqueness of the KKT multiplier, this furthermore implies that part (ii) holds. □ 



Lemma 4.1. Suppose that Assumptions \2. 1\ 13. l\ and \4. 1 \ hold. Then 

a) 11411 = <9(ii4n 2 ) + <wfr), 11411 2 = o(ii4n 4 ) + owp, \\h k \\ = o(\\d k \\ z ) + o(<p(x k r). 

(ii) It £ L(x k ) £ Io(x*)for k large enough. 

proof: (i) In view of F (x k + dt.) = <9(||c/qII 2 X the proof is elementary from (TTZt and Lemma l3~3T i). 

(ii) For i t Iq(x*), we have f(x*) < 0. Since lim(x*, 4) - (x*,0), there exists a constant £ > such that 

k— >oo 

f(x k ) < — £ < for k large enough. Moreover, it holds that f(x k ) + gi(x k ) T dQ < — if < for k large enough, which 
implies that i £ L(^*), i.e., L(^) Q Io(x*). Furthermore, it follows from Theorem 14. H ii) that lim A k J+ = A* > 0, i.e., 
A k i+ > and 7+ C L(x k ) hold for k sufficiently large. □ 

It is well-known that the strict complementarity condition (i.e., I* = Io(x*)) is very important to ensure L(x k ) = 
Io(x') holds, however, this condition is hard to verify in practice. In our paper, by the strong second-order sufficient 
conditions, we only need L(x k ) C Iq(x*). 

To ensure the steplength t\ = 1 for k large enough without the strict complementary assumption, an additional 
assumption is necessary. 

Assumption 4.2. Suppose that the KKT pair (x* , A*) and matrix satisfy 

\\(V 2 xx L c (x'^*)-H k )d k \\ = o(\\d k \\), 
where V 2 xx L c (x* , A*) = V 2 F c (x*) + £ A]V 2 fix") = V 2 L(x*,p*). 

iel 

10 



Theorem 4.2. Suppose that Assumptions \2.1\ I3.il \4.1\ and \4.2\ hold. Then the inequalities in ( 1751 ) always hold for 
t — 1 and k large enough. 

proof: We assume that t = 1 and k large enough in the whole process of proof. First of all, we discuss the second and 
the last inequalities of ( fT5l ). 

For i £ Iq(x*), i.e., fox*) < 0. In view of (x k , d k Q , d k 2 ,ip{x k )) — > (x*, 0,0,0) (k — > oo), we have d k - d k + d k —> 
(£ — > oo). So we can conclude that the second inequalities and the last inequality of ( TT3T > are both satisfied. 

For i 6 /o(x*), it holds that fi(x*) = 0. On one hand, since lim fox*) = fox*) = and lim ip{x k ) = as well as (01, 
it follows that -* and = o(\fox k ) + gjCJC*) 7 ^!) + o(|l4ll)- On the other hand, we have from (G2) and Lemma 



gi {x k ) T d k 2 = -n^ir - vi^r - Mx k + d k ) + /«(**) + gi(^) r 4 + + s«(**) r 4D + 0(ii4n 3 ) + o^n- 

(22) 

Then we obtain by Taylor expansion and (1221 

-H4H T - L/K**) + *i(**) T 4l + ofl/K**) + *i(**) T 4D + 0(\\d k \\\ if i e /(,(**) n r ; 
+ d*) = - — - ip(x k T + f(x k ) - \fi(x k ) + gi (x k ) T d k \ + oQfox*) + ft(JK*) r <#) (23) 
+0(||4|| 3 ) + ofaC**) 0- ) if i e /(**) n 7+. 
By t e (2, 3), the first equality of (l23l implies that + d*) < for all i e Iq(x*) n i.e., the second inequalities 
of (TT~5T > hold for i e Iq(x*) n 7~ and the third inequality of Sl5[ holds. 

Again from (1231 . t e (2, 3) as well as a e (0, i), we have for i e Io(x*) n 7 + 

Mx k + d k ) - max{0, (pCjc*) - a(\\d k \\ T + qtff)} = -(1 - <*)(|k2*|r + tp{x k Y) - \fox k ) + gi (x k ) T d k \ 

+oQfox k ) + gi (x k ) T d k \) + o{<p{x k Y) + 0(||d*I| 3 ) 
< 0. 

Summarizing the above analysis, we have proved that the second and the last inequalities of (TT3T > are satisfied for t = 1 
and k large enough. 

From now on, we will show that the first inequality of (fT~5T > holds. First of all, by Taylor expansion and Lemma 
14. Il l), we have 

A k = VF c (x k ) T d k + Ud k ) T V 2 F c (x k )d k - aVF c (x k ) T d k -p(l - a)w{x k ) e + o(\\d k \\ 2 ) 

(24) 

= VF c (x k ) T (d k + d k ) + ±(4) r V 2 F c (**)4 - aVF c {x k ) T d k -p(l - ar)y>(jc*) 8 + odl^ll 2 ) + ofaC**)""). 
Then we get by the KKT conditions 0) and Lemma [4~Tl i) 

VF c (x k ) T (d k + d k ) = -(d k ) T H k d k - 2 ^(**) r (4 + 4) + oGldgll 2 ) + o{<p{x k Y). (25) 

For i e L(**) c 7 (x*), it follows that fox*) + g i {x k j T d k = 0. From (23) and LemmagJli) as well as y>(**) = o^OY), 
we have 



fi{x* + d k ) 



-||d*ir - + odldgll 2 ) + o(y(*T), i e !<**); 

/■(x*) + ft (je*) r cd* + 4> + i(4) r v 2 /;(x*)4 + (ii4n 2 ) + o(tf(x k r), 
ii 



which further imply that 

- 2 A k g i (x k f(d k +d^ = 2 A k fi(x k ) + \ 2 4(4) r V 2 /i(^)4 + o(||4l| 2 ) + 0(^r). (26) 
Substituting (|26] > into d25l >. we have 

vf c (*y « + 4) = -(d k fH k d k + 2 + i(4) r (v^L c (*\ a*) - v 2 f c (**))4 + (ii4n 2 ) + (w*r), 

ieL(x k ) 

which combined with ( 1241 gives 
A, = -(d k ) T H k d k + 2 + |(4) r V 2 t L c (x*, - aVFctffdl - p(l - a)^(^) 9 + (||4|| 2 ) + O^jk*)^ 

ieL(x*) 

= (a - ±)(d k ) T H k d k + (l-a) 2 + ( V L^(**> ^) - ff t ) ^ 

- P (i - aMi 1 / + (|i4n 2 ) + OiytfY). 

Due to A k f(x k ) < and or e (0, i) as well as < cr, it follows from Assumptions 13 . 1 1 and !4.2l that 

A k < (a - ^)a\\d k \\ 2 + o(\\d k \\ 2 ) -p(l - + < 0, 

i.e., the first inequality of (15[ holds. The whole proof is completed. □ 

Theorem 4.3. Under all above mentioned assumptions, ip(x k+l ) = after a finite number of iterations, i.e., x k+l 6 £2 + 
for k large enough. 

According to Theorem 14. 2 1 and (1231 . the above theorem holds directly. Moreover, based on Theorems 14 . 2 1 and 14 . 3 1 
the superlinear convergence of Algorithm 12. H is given in Theorem l4.4l bv Theorem 2.2.3 in JitI . 



Theorem 4.4. Suppose that Assumptions \2.1\ 15. 71 14. 1 1 and \4.2\ are all satisfied. Then Alsorithm \2A\ is superlinearly 



convergent, i.e., \\xr +1 - x*\\ = o(\\x k - x* 



5. Numerical experiments 



191 



In this section, in order to show the effectiveness of our proposed algorithm, some classical problems in yj 
are tested and the corresponding comparative numerical results are reported in the following parts. The algorithm is 
implemented by using MATLAB R2008a on Windows XP platform, and on a PC with 2.53GHz CPU. 

During the numerical experiments, the identity matrix E„ is selected as the initial Lagrangian Hessian, and the 



approximation Hessian matrix H k is updated by BFGS formula [20] 



where 



H k s k (s k ) T H k fif) 7 
H ^ =H k - / + (k > 0), 

is")' H k s K {s K Yy K 



s k = x k+i _ x k^ =y k + ak ( 7kS k + A k A T k s k ), y k = min{||4ll 2 , K e (0, 1)}, A k = (gi(x k ), i e L(x k )), 

y k = V x L Ck (x k+ \A k ) - W x L Ct (x k ,A k ), W x L Ck (x k ,A k ) = VFJ^) + 2 A k gi (x k ), 

iel 
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and 



at = i 



0. 
1, 

1 + 



y t \\s t \\ 2 Hs k ) T A k (A t ) T s k 



if (/)V > fi\\A\ 2 , V e (0, 1), 

if < CS*)V < yUll/H 2 , 

otherwise. 



The parameters are selected as follows 

a = a = ri = K = n = c-i = 0.5, = 0.4, cr = 0.6, p = 2, t = 2.5, e = 0.5 3 , p = 2, y = 2, y = 1. 

The algorithm stops if the termination criterions || c/^ ||<= e and <p(x) = are both satisfied. 
First of all, some notations used in the following tables are defined in Table Q] 

Table 1 : Definitions of some notations 



Prob 


The number of test problem in IT81. 


NfO 


The number of objective function evaluations. 


71 


The number of variables of test problem. 


Nf 


The number of all constraint functions evaluations. 


hlh 


The number of equality and inequality constraints,respectively. 


Fv 


The objective function value at the final iteration point. 


Nio 


The number of iterations out of the feasible set. 


CPU 


The CPU time (second). 


Nii 


The number of iterations within the feasible set. 




The information is not given in the corresponding references. 


Ni 


The total number of iterations, i.e., Ni=Nio+Nii. 







In order to show the computational efficiency of Algorithm [2J] (shorted by ALGQ |2~D . which is compared with 
other types of algorithms, including SQP algorithms and systems of linear equations (SLE) algorithms. The statistics 
of these algorithms are given in Table [2] The "Feasible" (or "Infeasible") in Table [2] means that the initial iteration 
point have to be feasible (or can be chosen arbitrarily) for the solving problem. 



Table 2: Description of some comparative algorithms 



Author 



Types of algorithm (shorted by) Types of solving problem 



Jin and Wang 
Qi and Yang [2] 
Wang, Chen and He [22] 
Guo [12] 

Jian, Ke, Zheng and Tang ]23] 
Yang, Li and Qi [24] 
Gu and Zhu [£| 

Gill, Murray and Saunders \25] 



Feasible SQP (JW-FSQP) 
Infeasible SQP (QY-IFSQP) 
Infeasible SLE (WCH-IPSLE) 
Infeasible SQP (G-IFSQP) 
Infeasible SQP (JKZT-IFSQP) 
Feasible SLE (YLQ-FSLE) 
Infeasible SQP (GZ-IFSQP) 
SNOPT 



nonlinear inequality constrained programming 
nonlinear inequality constrained programming 
nonlinear equality and inequality constrained programming 
nonlinear inequality constrained programming 
nonlinear inequality constrained programming 
nonlinear inequality constrained programming 
nonlinear equality and inequality constrained programming 
nonlinear equality and inequality constrained programming 



In Table[3] we compare the number of Ni and Fv required by ALGO l2~T1 with those required by JW-FSQP. The test 

n 

problems are chosen from [18], and initial iteration points are all feasible except Prob 030. The optimality tolerance 
is the same as in B21I1 . The results in Table [3] show that the number of iterations of ALGO l2~T1 is much smaller than 
that of JW-FSQP for most test problems. From the viewpoints of Ni and Fv, we can conclude that ALGO l2~T1 is more 
effective than JW-FSQP. 
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Table 3: Comparative numerical results of ALGO 2.1 and JW-FSQP 



Prob 


nl\h\l\h\ 


ALGO 2.1 






JW-FSQP 








Ni 


Fv 


CPU 


Ni 


Fv 


012 


2/1/0 


7 


-3.000000000E + 01 


0.02 


18 


-3.000000000E + 01 


024 


2/5/0 


11 


-1.000000000E + 00 


0.06 


12 


-9.999997439E - 01 


029 


3/1/0 


10 


-2.262741700E + 01 


0.05 


16 


-2.262741700E + 01 


030 


3/7/0 


10 


4.016837909E - 18 


0.05 


14 


1.0000QOOOOE + 00 


031 


3/7/0 


13 


6.000000000E + 00 


0.08 


12 


6.000000007E + 00 


033 


3/6/0 


9 


-4.585785958E + 00 


0.05 


44 


-4.585786409E + 00 


043 


4/3/0 


10 


-4.400000000E + 01 


0.05 


24 


-4.400000000E + 01 


076 


4/7/0 


9 


-4.681818182E + 00 


0.05 


9 


-4.681818182E + 00 


100 


7/4/0 


15 


6.825663841E + 02 


0.11 


42 


6.806300574E + 02 



In Table |H we further compare the number of Ni, NfO, Nf and Fv required by ALGO 12.11 with those required 
by QY-IFSQP. The optimality criterions and the starting iteration points for the test problems are the same as in [fl. 
"Point (a)" (or "Point (b)") in Table |4]denotes that the corresponding initial point is "feasible" (or "infeasible"). Note 
that Nf refers to the number of evaluations of fj in |2|], however, Nf denotes the number of all constraint functions 
evaluations in our paper. The Ni column is displayed as the total number of iterations. Only if the initial iteration 
points are chosen as (a), Ni=Nii, otherwise, Ni=Nio+Nii. For example, "5+12" means that the algorithm generates a 
feasible point after five iterations, and after another twelve iterations the algorithm produces an approximately optimal 
solution. For the test problems, from the viewpoints of Ni and NfO, the results show that ALGO l2~Tl is obviously better 
than QY-IFSQP for Prob 034, 043, 065 and 100 for point (a). For point (b), again from the viewpoints of Ni, NfO and 
Fv, ALG I2.1l is competitive with QY-IFSQP for most of test problems except Prob. 037 and 043. 

Note that all test problems in Tables [3] and [4] only have inequality constraints. In order to show the performance 
of ALGO |2~T1 for solving problems with equality constraints, ALGO l2~T1 is further compared with WCH-IFSLE, GZ- 
IFSQP and SNOPT, respectively. The test problems and stopping criterions as well as initial iteration points are the 
same as in II 220 and |5J, respectively. 

For comparing the performance of ALGO |3T| with WCH-IFSLE and GZ-IFSQP as well as SNOPT, we use 
performance profiles as described in Dolan and More's paper 112611 . Our profiles for figures are based on the number of 
iterations. The function p{f) is the (cumulative) distribution function for the performance ratio within a factor t g R. 
The value of p(r) is the probability that the solver will win over the rest of the solvers. The corresponding results 
of performance are shown in Figure [1] From Figure [1] it is obviously that the performance of ALGO |2~T1 is better 
than that of WCH-IFSLE, i.e., ALGO |2~T1 has the most wins compare with WCH-SLE. Moreover, our algorithm is 
competitive with SNOPT (which is a well-known SQP algorithm for solving nonlinear constrained programming) 
although the performance of GZ-IFSQP is better than ALGQ|2~T1 

Note that the above test problems are relatively small. In order to show the more clearly effectiveness of ALGO 
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Table 4: Comparative numerical results of ALGO 2.1 and QY-IFSQP 



Prob 


nl\h\l\h\ 


Point 




Ni 


NfO 


Nf 


Fv 


CPU 


034 


3/8/0 


(a) 


ALGO 2.1 


24 


25 


443 


-0.83403244521568 


0.17 








QY-IFSQP 


34 


161 


162 


-0.83403244524796 


— 






(b) 


ALGO 2.1 


5+12 


18 


374 


-0.83403244522367 


0.13 








QY-IFSQP 


13 


32 


33 


-0.83403244526530 


- 


035 


3/4/0 


(a) 


ALGO 2.1 


12 


13 


119 


0.11111111111111 


0.05 








QY-IFSQP 


10 


46 


46 


0.11111111111111 


— 






(b) 


ALGO 2.1 


1+9 


11 


84 


0.11111111111111 


0.03 








QY-IFSQP 


9 


11 


12 


0.11111111111111 


- 


036 


3/7/0 


(a) 


ALGO 2.1 


7 


8 


114 


-3299.99999999996 


0.02 








QY-IFSQP 


3 


7 


7 


-3300 


- 






(b) 


ALGO 2.1 


1+4 


6 


77 


-3299.99999999997 


0.03 








QY-IFSQP 


5 


10 


10 


-3300 


- 


037 


3/8/0 


(a) 


ALGO 2.1 


23 


24 


467 


-3455.999999999965 


0.17 








QY-IFSQP 


12 


34 


34 


-3456.000000000001 


- 






(b) 


ALGO 2.1 


34+35 


70 


1189 


-3455.999999999998 


0.58 








QY-IFSQP 


29 


113 


119 


-3456.000000000001 


- 


043 


4/3/0 


(a) 


ALGO 2.1 


12 


13 


81 


-44 


0.06 








QY-IFSQP 


14 


29 


29 


-44.00000000000001 








(b) 


ALGO 2.1 


67+8 


76 


1270 


-44 


0.56 








QY-IFSQP 


19 


60 


65 


-44.00000000000011 


- 


044 


4/10/0 


(a) 


ALGO 2.1 


20 


21 


448 


-14.99999999935652 


0.17 








QY-IFSQP 


7 


13 


13 


-15 


— 






(b) 


ALGO 2.1 


4+5 


10 


190 


-14.99999999999756 


0.09 








QY-IFSQP 


8 


18 


21 


-15 


— 


065 


3/7/0 


(a) 


ALGO 2.1 


8 


9 


126 


0.95352885680478 


0.03 








QY-IFSQP 


13 


38 


39 


0.95352885680478 








(b) 


ALGO 2.1 


1 + 13 


15 


217 


0.95352885680478 


0.11 








QY-IFSQP 


15 


41 


44 


0.95352885680188 




066 


3/8/0 


(a) 


ALGO 2.1 
QY-IFSQP 


10 

8 


11 

8 


175 

8 


0.51816327418156 
0.51816327418154 


0.06 






(b) 


ALGO 2.1 


2+13 


16 


252 


0.51816327418154 


0.11 








QY-IFSQP 


14 


26 


28 


0.51816327418153 




100 


7/4/0 


(a) 


ALGO 2.1 


20 


21 


193 


682.5663838261504 


0.14 








QY-IFSQP 


28 


96 


96 


680.6300573744018 








(b) 


ALGO 2.1 


7+14 


22 


258 


682.5663838261520 


0.16 








QY-IFSQP 


26 


90 


92 


680.6300573743961 
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Figure 1: The left figure shows the performance of ALGO 2.1 and WCH-IFSLE, the right figure shows the performance of ALGO 2.1 and GZ- 
IFSQP as well as SNOPT. 



Table 5: Comparative numerical results of ALGO 2.1, G-IFSQP, JKZT-IFSQP and YLQ-FSLE 



Prob 


nl\h\l\h\ 


Algorithm 


Nii 


NfO 


Nf 


Fv 


CPU 


Svanberc 


-10 10/30/0 


ALGO 2.1 


14 


15 


1140 


15.731517 


0.38 






G-IFSQP 


17 


18 


1200 


15.731517 


0.44 






JKZT-IFSQP 


28 


28 


1753 


15.731533 








YLQ-FSLE 


36 


227 


258 


15.731517 




Svanberc 


-30 30/90/0 


ALGO 2.1 


23 


24 


6210 


49.142526 


2.67 






G-IFSQP 


25 


26 


5490 


49.142526 


2.09 






JKZT-IFSQP 


27 


27 


4975 


49.142545 








YLQ-FSLE 


101 


777 


864 


49.142526 




Svanberc 


-50 50/150/0 


ALGO 2.1 


29 


30 


13050 


82.581912 


5.59 






G-IFSQP 


33 


34 


11550 


82.581912 


5.95 






JKZT-IFSQP 


37 


37 


11762 


82.581928 








YLQ-FSLE 


108 


881 


968 


82.581912 




Svanberc 


-80 80/240/0 


ALGO 2.1 


38 


39 


33120 


132.749819 


13.80 






G-IFSQP 


42 


43 


24720 


132.749819 


15.38 






JKZT-IFSQP 


47 


47 


24100 


132.749830 








YLQ-FSLE 


190 


1666 


1835 


132.749819 




Svanberc 


-100 100/300/0 


ALGO 2.1 


42 


43 


43200 


166.197171 


23.09 






G-IFSQP 


55 


56 


39600 


166.197171 


30.53 






JKZT-IFSQP 


46 


46 


27880 


166.197199 








YLQ-FSLE 


178 


1628 


1782 


166.197171 
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12. ll for solving some large scale problems, the "Svanberg" problems are tested, which are selected from CUTE II 1911 . 
The corresponding results are given in Tables |5]and|6] In Table|5] the performance of ALGO l2~T1 is compared with G- 
IFSQP, JKZT-IFSQP and YLQ-FSLE, respectively. The initial iteration points are feasible and the stopping criterions 
are the same as that reported in J12I . From the results in Table |5j in viewpoints of Nil and NFO, it follows that 
ALGOl2~T1is more effective than G-IFSQP and JKZT-SQP as well as YLQ-SLE for solving "Svanberg" problems, 
respectively. 



Table 6: Comparative numerical results of ALGO 2.1 and G-IFSQP 



Prob 


nl\h\l\h\ 


Algorithm 


Nio 


Nii 


NfO 


Nf 


Fv 


CPU 


Svanberg- 10 


10/30/0 


ALGO 2.1 


3 


15 


19 


1290 


15.731517 


0.42 






G-IFSQP 


3 


15 


19 


1320 


15.731517 


0.73 


Svanberg-20 


20/60/0 


ALGO 2.1 


4 


21 


26 


4560 


32.427932 


1.53 






G-IFSQP 


4 


22 


27 


4380 


32.427932 


1.69 


Svanberg-30 


30/90/0 


ALGO 2.1 


3 


23 


27 


6570 


49.142526 


3.06 






G-IFSQP 


3 


25 


29 


6480 


49.142526 


2.58 


Svanberg-40 


40/120/0 


ALGO 2.1 


3 


23 


27 


9840 


65.861140 


3.75 






G-IFSQP 


3 


28 


32 


9480 


65.861140 


3.83 


Svanberg-50 


50/150/0 


ALGO 2.1 


5 


36 


42 


23700 


82.581912 


8.27 






G-IFSQP 


11 


30 


42 


16050 


82.581912 


8.16 


Svanberg-80 


80/240/0 


ALGO 2.1 


5 


81 


87 


101280 


132.749819 


34.11 






G-IFSQP 


2 


48 


51 


28320 


132.749819 


18.42 


Svanberg- 100 


100/300/0 


ALGO 2.1 


3 


43 


47 


43500 


166.197171 


25.58 






G-IFSQP 


3 


52 


56 


39900 


166.197171 


31.05 


Svanberg- 150 


150/450/0 


ALGO 2.1 


40 


44 


85 


166500 


249.818369 


112.69 






G-IFSQP 


34 


59 


94 


153000 


249.818369 


127.84 


Svanberg-200 


200/600/0 


ALGO 2.1 


3 


84 


88 


232800 


333.441310 


226.14 






G-IFSQP 


4 


94 


99 


148200 


333.441310 


307.72 



Moreover, ALGO l2~Tl is compared with G-IFSQP for "Svanberg" problems with infeasible initial iteration point, 
i.e. x° = (10, . . . , 10) r . The optimality thresholds are the same as in II 1211 . and the comparative results are given in 
Table [6] For Prob Svanberg-20, Svanberg-30 and Svanberg-40, although Nio and Fv of ALGO |2~T1 are the same as 
that of G-IFSQP, Nii of ALGO|XT]is less than that of G-IFSQP. For Prob Svanberg-50 and Svanberg-200, ALGO EH 
can enter into the feasible region more quickly than G-SQP. In view of Nio, Nii and Fv, it holds that ALGO 12.11 is 
more competitive than G-IFSQP. Furthermore, the comparative results of ALGO |2~T1 and G-IFSQP in Tables [5] and [6] 
further imply that the efficiency of MQSSFD is higher than that of MSSFD. And this also proves that the conclusions 
in J14I1 is correct. 
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6. Conclusions 



In this paper, inspired by the ideas in ||8j,[12|], an improved SQP algorithm with arbitrary initial iteration point for 
solving problem ([TJ is proposed. Firstly, problem (0 is equivalently transformed into an associated simpler problem 
©. At each iteration, the search direction is generated by solving an always QP subproblem and one (or two) SLE (s). 
The two SLEs have the same coefficient matrices. After a finite number of iterations, the iteration points always lie in 
the feasible region of problem ©, and we only need to solve the one SLE. In the process of iteration, the feasibility 
of the iteration points is monotone increasing. Under some mild assumptions without the strict complementary, our 
algorithm possesses global and superlinear convergence. Some comparative numerical results in Section |5]show that 
our algorithm is effective and promising. 
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